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This study employed a web-based survey investigating graduate students' 
perceptions of effectiveness of various learning activities in an online teacher 
education course designed to teach instructional strategies. Learner-centered 
evaluation allows for insights into the teaching and learning process, and learner 
satisfaction is particularly critical in determining quality in distance education. The 
findings wouid inform a redesign of the course with the goai to enhance learning, 
using students as evaluators. The students' ratings and comments of course 
activities are discussed, and impiications related to course redesign are examined. 

Nearly 4 million students were enrolled in an online course in higher 
education in the United States during the Fall 2007 semester (Allen & Seaman, 
2008). With the burgeoning growth and demand for collegiate online courses, it is 
imperative that we respond to the needs of online learners to ensure academic 
success (Moore, Sener, & Fetzner, 2006). Studies that investigate and confirm that 
individual distance education courses support rigorous learning outcomes, 
effectively use available technology to improve pedagogy, and provide student 
satisfaction are essential to the scholarship of teaching and learning. This article 
recounts one such study focused on student 

learning and, in doing so, documents learning Studies that investigate and 

effectiveness in order to make the teaching confirm that individual 

and learning process public and to advance distance education courses 

the practice of teaching (Flutchings, Babb, & support rigorous learning 

Bjork, 2002). outcomes, effectively use 

The purpose of the study reported available technology to 

herein was to replicate and substantiate the improve pedagogy, and 

results of a previous study conducted by the provide student satisfaction 

researcher (Lee, 2009) to inform a redesign of essential to the 

a course. The intent was to investigate scholarship of teaching and 

graduate students' perceptions of the learning, 

effectiveness of various learning activities in 

an online teacher education course designed to teach instructional strategies for 
improving secondary teaching. The findings informed a redesign of the course with 
the goal of enhancing learning, using students as evaluators (Calloway, 2008; 
Cuthrell & Lyon, 2007). We know that learner- centered evaluation allows for 
insights into the teaching and learning process (Cerbin, 1995), and that learner 
satisfaction is particularly critical in determining quality in distance education 
(Belfer, 2000). This study additionally addressed Shulman's three rationales for the 
scholarship of teaching and learning: professionalism, pragmatism, and policy 
(2000). As a teacher educator, I have the professional responsibility to ensure that 
an online course designed to teach and model effective instructional strategies does 
what it purports to do. Online courses in teacher education have the added 
responsibility of modeling "best practices" in online design and online facilitation due 
to a substantial increase of online courses offered in K-12 schools. Some of these 
teachers will likely become online facilitators themselves, as experts predict that 
online learning in K-12 schools will accelerate (Picciano & Seaman, 2009). 
Furthermore, engaging in such inquiry and documenting the journey facilitates the 
sharing of our craft, resulting in a body of work that "becomes public, peer-reviewed 
and critiqued, and exchanged with other members of our professional communities 
so they, in turn, can build on our work" (Shulman, 2000, 49). Surveying the 
graduate students themselves to investigate their perceptions of the effectiveness of 
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the online learning activities was a pragmatic approach designed to inform "efforts 
in the design and adaptation of teaching in the interests of student learning" 
(Shulman, 2000, 49). Also, this research provides a "policy" rationale, responding to 
standards of quality assurance in teacher education (National Council for 
Accreditation of Teacher Education, 2008) and distance education (Wang, 2006). 

Method 


Participants 

At the end of the semester and after course grades had been calculated, I 
recruited participants who had been enrolled as students in the online graduate 
course, using the class roster with university e-mail addresses. An e-mailed 
invitation to 26 individuals yielded 24 participants. The invitation included the 
purpose of the research survey, the estimated time to complete it, explanations 
related to informed consent and confidentiality, the Institutional Review Board 
approval number, and a two-week deadline for completion. Table 1 summarizes the 
demographic characteristics of the participants. 

Table 1: Demographic Characteristics of Participants (N = 24) 


Characteristic 

N 

% 

Age at time of survey (years) 



23-27 

7 

29.2 

28-33 

11 

45.8 

45-47 

2 

8.3 

50-55 

4 

16.7 

Gender 



Female 

16 

67 

Male 

8 

33 

Ethnicity 



White 

18 

75 

Hispanic 

4 

16.7 

Asian 

2 

8.3 


Measures 

A survey was employed as the research tool to examine the students' 
perceptions of the effectiveness of the online activities, affording a process of 
examination and reflection, "viewing teaching and research as fundamentally 
dialogical activities" (Ritchie & Goodburn, 1996, p. 76). The online course survey 
was created using SPSS mrinterview (4.0), a browser-based authoring tool. This 
sophisticated tool afforded respondent data to be exported directly into SPSS, a 
data analysis software program. The survey was designed to gather basic 
demographic data on the participants and determine the graduate students' 
perceptions of the effectiveness of the online instructional activities. A categoric grid 
allowed respondents to quickly and easily rate a comprehensive list of the course 
activities. Several open-ended questions allowed respondents to personally 
comment on various aspects of the course activities. 

Results 

Participants rated each of the weekly activities in response to the question, 
"How effective were these learning activities in facilitating your learning of 
instructional strategies?" Point values for the responses were assigned as follows: 
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excellent (1), good (2), fair (3), and poor (4). Means and standard deviations were 
calculated for each activity. Table 2 presents the graduate students' perceived 
effectiveness of the online activities from most effective to least effective, as 
demonstrated by the mean and standard deviation scores. 

Table 2: Rating of Online Activities by Number, Mean and Standard 
Deviation, N = 24 


Activity 

M 

SD 

Field Experience Portfolio: Construct lesson plan for video teach 

1.38 

.65 

Field Experience Portfolio: Construct portfolio notebook of field 
practicum 

1.38 

.58 

Performance Assessment: Construct extended-type performance task 
and assessment rubric In your content area 

1.50 

.98 

Field Experience Portfolio: Reflect on video teach, write reflection, and 
upload to Live Text 

1.50 

.78 

Field Experience Portfolio: Record video teach and burn onto CD 

1.50 

.78 

Field Experience Portfolio: Conduct teacher Interview 

1.54 

.88 

Field Experience Portfolio: Analyze classroom observations 

1.54 

.88 

Mental Models About Teaching: View "First Day of Class" video and 
categorize teaching behaviors 

1.62 

.77 

Active Learning: Watch "Jerry Seinfeld" and "The Mirror Flas Two 
Faces." Discuss effective and Ineffective practices In forum 

1.63 

1.14 

Questioning Styles and Strategies: Discuss "Common questioning 
errors" 

1.63 

.77 

Building a Learning Community: Introduce self In "All About Me." 
Include photo and welcome to classmates. 

1.67 

.70 

Cooperative Learning: Read, map, and summarize the 5 essential 
elements of cooperative learning 

1.67 

.92 

Mental Models About Teaching: Discuss "Seven Myths of Learning" 

1.71 

.75 

Motivation: Motivation Factors: Discuss Influence of race, SES, 
linguistic ability, religion, etc. 

1.71 

1.12 

Learning Styles: Complete Index of Learning Styles Questionnaire and 
analyze results 

1.75 

.74 

Motivation: Cognitive Interactions: Post thought-provoking questions 
and responses to "Concepts of Ability and Motivation" 

1.75 

.85 

Assessment: Instruction: Massaging the TEKS 

1.79 

.78 

Learning Styles: Claim research article and post responses to the 
multicultural perspective discussion forum 

1.83 

.92 

Mental Models About Teaching: Create a broadcast letter 

1.87 

.80 

Engaging Students: Create magazine cover as summary for "Making 
Learning Real: Engaging Students In Content" 

1.87 

.85 

Engaging Students: Watch Newscast video clip and discuss 
advantages and disadvantages of strategy 

1.88 

.99 

Culturally Responsive Pedagogy: Create ppt. presentation on Ch. 12 
"Addressing the Academic Needs of Immigrant Students" 

1.92 

.93 

Cooperative Learning: View "Working Together" clips on Teaching 
Alive CD and answer guided reading 

1.96 

.96 

Quiz: Ch. 3: Lesson Planning and Assessment Obiectives 

1.96 

.81 

Building a Learning Community: Post personal goals using SMART 
criteria 

1.96 

.91 

Building a Learning Community: Post and discuss "Online Student 
Readiness Survey" results 

2.00 

.72 

Field Experience Portfolio: Analyze a textbook 

2.00 

.89 

Personalizing Culture: Discuss personalization of culture. 

2.00 

1.22 

Student-Centered Instruction: Watch "Good Morning Miss Tolliver" 

2.04 

1.12 
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video ciip and discuss in forum 



Assessment: Students: Read "Lessons at the Kitchen Tabie" and 
discuss deficit thinking and funds of knowiedge 

2.04 

1.33 

Personaiizing Cuiture: Compiete jigsaw of "Personaiizing Cuiture 
through Anthropoiogicai and Educationai Perspectives." 

2.08 

1.10 

Questioning Styies and Strategies: Observe questioning strategies in 
"Teaching Aiive" CD 

2.08 

1.14 

Student-Centered Instruction: Choose graphic organizer to 
summarize Ch. 7 "Refiective Teaching and Learning: Students as 
Stakehoiders" 

2.08 

1.21 

Quiz: Ch. 5: Assessment During Instruction 

2.12 

.99 

Cuituraiiy Responsive Pedagogy: Compiete guided reading questions 
to Ch. 10 "Effective Muiticuiturai Teaching Practices" 

2.12 

.85 

Assessment: Students: Take quiz on Assessment text Chapters 1 and 
2: Breadth of Assessment and Learning About Pupiis Eariy 

2.17 

.96 

Learning Styies: Compare resuits of 2 different oniine LS inventories 

2.17 

1.31 

Student-Centered Instruction: Compiete graphic organizer Ch. 5 
"Interactive Practice for Learning: Beyond Driii" 

2.21 

1.38 

Active Learning: Identify the ABCCD components of an objective 

2.29 

1.27 

Performance Assessment: Take quiz over Ch. 8 "Performance 
Assessment" 

2.29 

.91 

Cooperative Learning: | iqsaw Group Debriefing: How effective? 

2.29 

1.23 

Buiiding a Learning Community: Take "Course Information & 
Syiiabus" quiz 

2.42 

.93 

Personaiizing Cuiture: View "Teaching Aiive" video ciip on 
contextuaiization and describe a unit in content area that exempiifies 
"meaning making" 

2.46 

1.62 

Cooperative Learning: Just for Fun: Watch "Emperor's New Groove" 
video ciip and discuss the tenets of cooperative iearning 

2.58 

1.59 

Cooperative Learning: J ust for Fun: What wouid J ohnson and J ohnson 
say? 

2.67 

1.47 


Discussion 

Context of Online Instructional Activities 

This teacher education course, Strategies for Improving Secondary 
Teaching, is required for a Master's degree in education and/or post-baccaiaureate 
teacher certification, and the course is aiso a prerequisite for student teaching. The 
course utiiizes a 30-hour fieid experience modei, aiiowing graduate students the 
opportunity to bridge theory and practice (Brandsford, Peiiegrino, & Donovan, 
1999). Units of instruction are organized into weekiy iearning moduies (e.g., 
buiiding a iearning community, mentai modeis of teaching, cooperative iearning, 
etc.): to scaffoid the iearning, aii the week's activities are bound in one iocation. 
The findings of this foiiow-up study were congruent with the findings of the previous 
study. Data from both surveys supported simiiar generai categories of most 
effective and ieast effective activities. 

Most Effective Activities 

In response to the open-ended survey question asking the students to 
identify the most effective iearning activity, the fieid experience activities were 
rated among the highest. As two students reported: 

• "The portfoiio process— I reaiiy enjoyed all the aspects of the 
classroom and my observation. The experience really showed 
me that I want to be in a classroom and I thought it was 
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important to see other subjects beyond your content area. I 
wish I couid have gone the whole semester; I did not want to 
leave. " 

• "1 had a wonderful time with my field experience. I honestly 
wish it were longer than 30 hours. I wish it were more of an 
every week thing. It allowed me to look into the lives of 
teachers and see what their day-to-day activities were 
actually like. They allowed me to interject whenever I wanted 
and made me feel very comfortable. It also led to a possible 
job. The district wants to hire me on in a new school they are 
opening. Thanks to my field experience, I was given a 
wonderful opportunity!" 

The students' field experience portfolio, submitted at the end of the 
semester, included a log documenting 30 hours in the field, a teacher interview, 
textbook analysis, classroom observations in three different content areas, a lesson 
plan (created in concert with the cooperating teacher), a "video teach" (videotaped 
teaching of the lesson plan), and video teach reflection. Other comments related to 
the value of the field experience included: 

• "I think that the video teach process was the best because 
you get to think about integrating learned methods and 
reflect on it. The reflection piece is vital because the camera 
does not lie. " 

• "The video teach was my favorite because it got me out of the 
house and into my field that I enjoy. I learned the most from 
the teacher interview; getting information from experienced 
teachers is priceless. " 

• "I really enjoyed the field experience. I worked with a great P. 

E. teacher, and was actually able to get in the classroom and 
work with the kids. I learn better by hands on activities, so 
this was very helpful to me. " 

Least Effective Activities 

In response to the open-ended survey question asking the students to 
identify the least effective learning activity, assignments and quizzes related to the 
weekly readings were among those most commonly identified. Several participants 
commented on their perceptions of the ineffectiveness of these learning activities: 

• "I have never been a strong reader, so the articles, review 
questions, and quizzes took a lot of time. " 

• "Questions from chapters; I just don't like those kinds of 
assignments. " 

• "I did not like reading text and taking quizzes or answering 
guided reading questions. I was not motivated; I had to work 
hard to remain focused. But I do understand that reading and 
Q&A is required. " 

Regarding the online multiple choice quizzes over the reading content, the 
following responses were noted: 

• "The chapter assessments, although usually straightforward, I 
am not a fan of the multiple-choice tests. I would much rather 
respond to questions in a written format. " 

• "I did not like the quizzes. " 

• "I didn't like the quizzes online. I took notes as I was reading 
and knew the info, but still didn't do very well on the quizzes. 
Assignments are a better way to test understanding. " 
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Suggestions for Improvement 

Several respondents offered suggestions for improving the course via an 
open comment section of the survey: 

• "Delete about 1/3 of the weekly assignments. I felt 
overwhelmed with the quantity of work. " 

• "Have less work in the week and more meaningful projects. " 

• "Instead of assigning readings, have the students find their 
own valid articles to summarize and discuss. Have them 
present their own vision of teaching, pedagogy, and 
strategy. " 

• "1 would love to have an actual chat session, like an active 
conversation but with time constraints and people's 
schedules, it's probably impossible. " 

Utilizing the student feedback collected from this survey and the previous 
one (Lee, 2009) provided valuable information on ways to improve the design of the 
course and the online learning experience for future students. 

Considerations for Redesign 

Although the overall high student ratings of the perceived effectiveness of 
the learning activities incorporated in Strategies for Improving Secondary Teaching 
indicate a strong original design, it is critical to examine, analyze, and improve upon 
the learning activities identified as needing improvement. Originally the course was 
co-created by a subject matter expert and an instructional designer, neither of 
whom had taught an online course. The logical "next step" was to investigate the 
course effectiveness through the lens of students who had completed the course 
and the faculty member who had taught it. 

Build on Strengths 

The field experience was perceived as the most valuable component of the 
course. In conjunction with students' suggestions for having fewer and more 
meaningful activities, decreasing the number of activities and integrating the course 
activities more closely with the field experience is warranted. Further study of the 
most "effective online strategies for integrating coursework and field experiences" 
(Knapczyk & Hew, 2007) will improve the value of several course activities. Dykman 
and Davis (2008) caution that -- 

it is difficult for an instructor to judge workload levels in an online 
course. There is a real tendency to overload the student with work 
to make sure that an online course, which is potentially visible to 
other faculty and administrators, has a level of content and rigor 
equivalent to a comparable conventional course (159). 

As the online facilitator of this course for several semesters and having 
graded the large quantity of course assignments, I agree that there are too many 
assignments. In his recommendation to "analyze and balance interactions" Hirumi 
warns that too many interactions can frustrate online learners, resulting in cognitive 
overload and an overwhelmed instructor (2003, p. 79). 

Integration & Effective Use and Modeling of Technology Tools 

Although this online course was offered through the university's open 
source course groupware, which offers built-in technologies such as a discussion 
board, digital drop box, and wiki space, integrating the use of the prolific and easy- 
to-use Web 2.0 tools is essential to model and practice the use of these innovative 
and 21®‘ Century technologies to enhance learning. For example, one of the 
suggestions for course improvement was to offer a chat, an activity resembling 
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"more of an active conversation." One way to address this issue of personaiization 
wouid be to utiiize Tokbox (http://www.tokbox.com), a free video chat and video 
emaii provider. Students and the instructor wouid be abie to video chat in "reai 
time" if scheduies aiiowed, and/or video emaiis couid be recorded and sent 
asynchronousiy. Griffiths & Graham (2009) recentiy found that using asynchronous 
video in an oniine course faciiitated a sense of personai connection and immediacy 
between instructor and students. Using asynchronous video as a vehicie through 
which students couid articuiate and demonstrate their understanding of course 
readings wouid be a viabie option to repiace the quizzes and/or chapter questions. 
Furthermore, aiiowing the students a choice of formats wouid support different 
iearning styies and the motivationai aspect of andragogy (Pew, 2007). 

Sound Pedagogical Design 

The overaii high ratings the students assigned to the various iearning 
activities may be attributed to the course designers' commitment to sound 
pedagogicai practice, as they foiiowed Newiin & Wang's (2002) recommendation 
that facuity in aii discipiines appiy American Association of Higher Education's Seven 
Principies of Good Practice in Undergraduate Education (1987) when designing 
oniine courses. The seven principies are: 

1. Encourage contact between students and facuity 

2. Deveiop reciprocity and cooperation among students 

3. Use active iearning techniques 

4. Give prompt feedback 

5. Emphasize time-on-task 

6. Communicate high expectations 

7. Respect diverse taients and ways of iearning 

These principies of "best practice" were incorporated into the design of the 
course. For exampie, contact between students and facuity was faciiitated by the 
creation of weekiy discussion board forums for diaiogue among students and the 
instructor focusing on the concepts being expiored. An "Assignment Ciarification" 
forum was created as a centrai iocation for students to pose questions or concerns 
about assignments, fieid experience requirements, and other course-reiated 

concerns. Numerous interactive activities and media were incorporated into the 

design of the course to meet the diverse iearning preferences of the students 
enroiied in the course. For exampie, video ciips modeiing the use of various 
instructionai strategies in the ciassroom were empioyed; routine weekiy activities 
inciuded student-created iearning artifacts as demonstration of their iearning. I 
communicated high expectations by providing 
performance assessment rubrics outiining 
specific assignment criteria and provided 

weekiy feedback to the students regarding 
their progress. 

Incorporating student survey data as 
formative feedback to improve a course 

design supports a student-centered iearning 
paradigm. Additionaiiy, integrating Web 2.0 technoiogies as powerfui iearning toois 
faciiitates the evoiution of effective oniine instruction and iearning. The venue of 
schoiarship of teaching and iearning offers those who are dedicated to the 
advancement of teaching and iearning a vehicie to document and share their 
research efforts and schoiariy work. "Schoiarship of teaching and iearning supports 
our individuai and professionai roies, our practicai responsibiiities to our students 
and institution, and our sociai and poiiticai obiigations to those that support and 
take responsibiiity for higher education" (Shuiman, 2000, p. 52). 


Incorporating student 
survey data as formative 
feedback to improve a 
course design supports a 
student-centered learning 
paradigm. 
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